The invention concerns a method for manipulation of at least one movable, natural object in a natural television picture, wherein the television picture is generated by one or more television cameras. The invention also concerns a method for generating at least one synthetic track in a television picture, wherein the synthetic track represents the path of a movable natural object in a natural television picture during a given period xcex8, and wherein there is employed a method for manipulation of the movable, natural object in the television picture. Finally, the invention concerns a system for implementation of the method for manipulating at least one movable, natural object in a natural television picture, wherein the television picture is generated by one or more television cameras, together with implementation of the method for generating a synthetic track in a television picture, wherein the synthetic track represents the path of a movable, natural object in the television picture during a given period xcex8, and wherein a method is employed for manipulating the movable, natural object in the television picture.
In television broadcasts where a movable, natural object plays a central part in the broadcast, it can often be difficult to follow the object or the movement thereof in the television picture. This is the case, e.g., in sports broadcasts from various ball games, such as football, handball, tennis, golf and ice hockey, where the picture format used, the background of the picture, colour, light conditions etc. can make it difficult to follow the object or the movement thereof. The object may also be invisible for a brief or a considerable period because it is masked by other objects in the picture. Such problems can be a factor in influencing the popularity of the television broadcasts, which in turn can have consequences for the sponsorsxe2x80x2 choice of programs or purchase of commercials in such programs, since the viewing figures are not as high as is desirable.
In order to make a game like ice hockey more attractive to the television medium, on the basis of the usual complaint about television coverage of ice hockey matches that the puck is difficult to follow, it has been proposed to make the ice hockey puck more clearly visible in the television picture by employing a special puck which is equipped with a number of infrared emitters driven by a battery provided in the puck. On the rink the puck is tracked by sensors placed along the edge of the rink, the sensors transmitting information to a camera which is connected to a computer. The information is fed to a data processing centre where a signal is generated which is processed graphically and introduced into the television picture. The puck can thereby be caused to change shape or colour. This system has been introduced by the company Fox Sports in the USA and is employed to represent the puck, e.g., surrounded by a shining halo in some colour or other and equipped with a coloured xe2x80x9ccomet tailxe2x80x9d when the puck is in motion, e.g. at a certain speed. The colours can, however, be altered according to the wishes of the producer. In a practical embodiment of this system which is called xe2x80x9cFoxTraxxe2x80x9d, 16 sensors are employed around the rink and two infrared cameras which also follow the puck. The processed representation of the puck is introduced into the television pictures which are recorded by the normal television cameras, and the special colour effects such as the halo and the xe2x80x9ccomet tailxe2x80x9d are overlaid the television signal with a delay which is not greater than between ⅙ and ⅓ of a second. xe2x80x9cFoxTraxxe2x80x9d is described in U.S. Pat. No. 5,564,698.
However, the system is complicated and offers limited opportunities for manipulation of the object, i.e. the picture of the puck. Another drawback is that the infrared emitters in the puck are driven by a battery which has a limited life, with a life of only 10 minutes having been quoted. This, however, is sufficient to allow the puck to be replaced during change of sides in a period of an ice hockey match, but makes a similar system less suitable in games where the object, i.e. the puck or ball, is in play for a longer period.
U.S. Pat. No. 4,675,816 (Brendon and Vinger) discloses a method for electronic localization of the ball in American football. The object of the method is to determine whether the ball has moved 10 yards forwards and provides simultaneous precise localization information about the ball and the possibility of positioning the football on the field. This method is regarded as an aid for spectators, officials and television. It comprises steps for providing a radio transmitter in the football and transferring radio signals from the football to a number of rotating receiver antennas, the antenna being directed towards the radio transmitter in such a manner that the radio waves supply accurate angular direction signals which can be used to calculate the position of the radio transmitter and the football on the field. These angular direction signals are supplied to a microprocessor which calculates the position of the transmitter and the football, with the use of a triangulation method which includes a computation stage which also indicates how far the ball has moved forwards, and the results of this computation stage can be shown on display units with a view to spectators and television viewers as well as being transferred to a control unit which is employed by the officials in order to monitor the course of the game.
From U.S. Pat. No. 5,138,322 (Nuttel) there is also known a system for continuous and precise measurement of the positions of a generally symmetrical object, such as a tennis ball, which is in motion in a predetermined three-dimensional area, such as a tennis court. In this case a number of antennas are employed which transmit radar signals to the three-dimensional area, reflected return signals from the ball being detected and compared with the transmitted signals for phase determination of the return signals, thus enabling unambiguous distances to the object or the ball to be determined. For this purpose a statistical method is employed in order to achieve an accurate determination of distance. The path of the object or the ball can be calculated simultaneously, and the system is calibrated by placing signal reflectors in different fixed positions on the court. A Doppler radar technique is employed.
Furthermore, in U.S. Pat. No. 5,346,210 (Utke and al.) there is disclosed an object localisation system, especially for localizing the ball in a special playing situation in American football. The system employs three sensors placed on one side of the field and a calibration source placed on the other side. The calibration source transmits an ultrasound signal which is received by the sensors and a ball marking unit which can be placed on the field instead of the ball also transmits an ultrasound signal which is received by the sensors, together with an RF signal which is received by the calibration source in order to switch it off. The sensors emit signals which are used by a processing unit to calculate time delays in order to determine the ball""s position. Alternatively, the ball marking unit may only transmit an RF signal which is received by the sensors which in turn emit signals which are processed in order to determine time delays between the receipt of the signals in the sensors. In addition, an automatic ball marking unit which is moved on a track is controlled by the processing unit in order to create automatically a visual representation of the position of the ball.
None of the above-mentioned, known systems, however, is particularly well suited to achieve full freedom to manipulate the picture of a natural object in a television picture, all requiring the use of relatively expensive and complicated systems. Nor are they suitable for all types of games or sports and moreover they appear to be substantially restricted to use in sporting events, but other forms of television broadcasts are, however, conceivable, where it will be equally interesting to be better able to visualise a movable object in a television picture with suitable detection and processing methods.
Consequently, it is a first object of the present invention to provide a system for manipulating the picture of at least one movable, natural object in a natural television picture in such a manner that the object""s position and movement are clearly visible in the television picture.
A second object of the present invention is to provide a synthetic representation of the natural object, thus enabling the synthetic representation to appear as a synthetic object in the television picture and to represent the natural object""s movement and position.
A third object of the present invention is to manipulate the synthetic object in the television picture with regard to shape and colour in such a manner that the viewer will have no trouble in following the object.
A fourth object of the present invention is to determine the path of a movable, natural object and to visualise this path in the form of a synthetic track for the object in the television picture.
Finally, it is an object of the present invention to provide a system which makes it possible to detect the movement of such a natural object and to process the detected data in order to generate and manipulate a representation of the movable object in a television picture as well as generating a synthetic track which represents the movement of the natural object in the television picture.
The above-mentioned objects and other advantages are achieved with a method which according to the invention is characterized by detecting the distance between the object and at least 2 fixed basic positions in a preselected x,y,z co-ordinate system at a time t, each basic position corresponding to a known position of a detector, determining an x,y,z co-ordinate for the object in the preselected x,y,z co-ordinate system at the time t, determining the distance between the camera""s lens centre and the object at the time t as an object vector in the preselected co-ordinate system, determining the television camera""s optical axis in the preselected co-ordinate system at the time t as a camera vector in the preselected co-ordinate system, determining a line from the television camera""s lens centre to the point of intersection between the edge of the generated television picture and the plane formed between the object vector and the camera vector at the time t as a zoom vector, the object vector being located between the camera vector and the zoom vector when the object is visible in the television picture at the time t, and when the object is not visible in the television picture at the time t the zoom vector is located between the camera vector and the object vector, and determining an X,Y position of the object referred to the television camera""s picture plane and the camera vector on the basis of the object vector and the camera vector at the time t, and if the object vector is located between the camera vector and the zoom vector, to insert a synthetic object in the X,Y position in the television picture at the time t, the synthetic object constituting a representation of the natural object recorded by the camera at the time t, or if the zoom vector is located between the camera vector and the object vector at the time t, to insert a symbol in the television picture, the symbol indicating the location of the X,Y position of the natural object outside the edge of the picture referred to the television camera""s picture plane and the camera vector.
The above-mentioned objects and advantages are further achieved with a method which according to the invention is characterized by calculating the path of the natural object on the basis of detected positions x,y,z for the natural object in a preselected x,y,z co-ordinate system at the time t, where txcex5xcex8, converting the detected positions at the time t to an X,Y position in the television camera""s picture plane at the time t, and generating the synthetic track in the television picture as the connecting line between all X,Y positions for the natural object in the picture planes of the natural television pictures which are recorded sequentially during the period xcex8.
The above-mentioned objects and advantages are further achieved by implementing the stated methods according to the invention with a system which according to the invention is characterized in that it comprises a transponder provided in the natural object and arranged to react to an optical, acoustic or electromagnetic signal received by the transponder with transmission of a response signal, at least one position module with at least 2 position detectors for transmitting optical, acoustic or electromagnetic signals and receiving response signals from the transponder and provided in respective basic positions in a preselected x,y,z co-ordinate system, together with a signal processor 1 arranged to determine the distance between a position detector and the object at a time t, a computing module connected with the signal processor and arranged to compute the x,y,z co-ordinates for the object in the preselected co-ordinate system at the time t and on the basis of the computed x,y,z co-ordinates for a number of times t to calculate a path for the object, together with an object vector given by the distance between the camera""s lens centre and the object at the time t, a camera vector given by the camera""s optical axis at the time t and a zoom vector between the camera""s lens centre and the point of intersection between the edge of the picture and the plane formed between the object vector and the camera vector at the time t, the object vector being either located between the camera vector and the zoom vector or the zoom vector between the camera vector and the object vector, a camera control system connected with the computing module and arranged to detect or generate values for the camera settings, and a manipulator module connected with the camera control system and the computing module and arranged to a) create a synthetic object in an X,Y position for the natural object in the recorded television picture at the time t, the synthetic object constituting a representation of the natural object recorded by the camera at the time t or b) create a symbol in the television picture, the symbol indicating the X,Y position for the natural object outside the edge of the picture at the time t, c) generate and select attributes for the synthetic object, or d) generate at the time t a synthetic track in a recorded or generated television picture, the synthetic track representing the path for the natural object during a period xcex8 before or up to the time t.